Contextual Control Policy Selection

Every autonomous agent operating in realistic settings must deal with incomplete state information. Sensory limitations, due to hardware constraints and/or limited interpretation algorithms, introduce hidden states which can prevent the acquisition of optimal control policies for a given task. This paper addresses this problem within the dynamical systems framework. The idea is to treat the agent in its environment as a dynamical system, and augment the original state space using contextual cues extracted empirically as the agent exercises existing control policies. Contex-tual cues are provided by the correlation between dynamic features of the agent-environment interaction and agent performance. In principle, the augmented state space allows the agent to learn to exploit control policies whenever they are likely to succeed, gather more information if the agent's context is ambiguous , and explore control alternatives in unfavorable contexts. Initial experiments involving an agent with impoverished sensing capabilities in a simulated, dynamic environment suggest that this approach might ooer a relatively simple way to extract meaningful context information from dynamical systems, and to enhance the agent's performance.